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Abstract 
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lems for free products G = A-kc B of groups A and B with amalgamation 
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generic stratum the conjugacy search problem is decidable under some 
reasonable assumptions about groups A, B, C . 
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1 Introduction 
1.1 Motivation 

This is the first paper in a series of four written on the Word and Conjugacy 
Problems in free products with amalgamation and HNN extensions. In this 
introduction we mention few results from the other parts as well when this 
improves the presentation of the main concepts. 

Free products with amalgamation and HNN extensions are among the most 
studied classical constructions in algorithmic and combinatorial group theory. 
Methods developed to study the Word and Conjugacy Problems in these groups 
became the classical models much imitated in other areas of group theory. We 
refer to Magnus, Karrass, and Solitar book [27| for amalgamated free products 
techniques and to Lyndon and Schupp book [22^ for HNN extensions. 

In 1971 Miller proved that the class of free products A*c B oi free groups 
A and B with amalgamation over a finitely generated subgroup C contains 
specimens with algorithmically undecidable conjugacy problem [2S^ . This re- 
markable result shows that the conjugacy problem can be surprisingly difficult 
even in groups whose structure we seem to understand well. In a few years more 
examples of HNN extensions with decidable word problem and undecidable con- 



2 



jugacy problem followed (see the book by Bokut and Kukin [5]). The striking 
undecidability results of this sort scared away any general research on the word 
and conjugacy problems in amalgamated free products and HNN extensions. 
The classical tools of amalgamated products have been abandoned and replaced 
by methods of hyperbohc groups [3 HSl 130] j or automatic groups US], or 
relatively hyperbolic groups [TH [33] • 

In this and the subsequent paper [llj we make an attempt to rehabilitate 
the classical algorithmic techniques to deal with amalgams. Our approach treats 
both decidable and undecidable cases simultaneously. We show that, despite the 
common belief, the Word and Conjugacy Problems in amalgamated free prod- 
ucts are generically easy and the classical algorithms are very fast on "most" 
or "typical" inputs. In fact, we analyze the computational complexity of even 
harder algorithmic problems which lately attracted much attention in cryptog- 
raphy (see O [261 [35], and surveys [HI [37]), the so-called Search Normal Form 
and Search Conjugacy Problems. Our analysis is based on recent ideas of strat- 
ification [31] and generic complexity [HI [21] , which we briefly discuss below. 

1.2 Stratification of the set of inputs 

We start with a general formulation of our approach to algorithmic problems and 
then specify it to algorithmic problems in groups. We follow the book Compu- 
tational Complexity of Papadimitriou [34j for our conventions on computational 
complexity. 

Let M be a set with a fixed size function size : M R^q a-nd A a partial 
algorithm with inputs from M. Denote by Dom^ C M the set of inputs on 
which A halts. For w G Dom^ by Tj^{w) we denote the number of steps required 
for the algorithm A to halt on the input w. If / : K^o — > IR^o is a standard 
complexity time bound, say f{x) = x", or f{x) = n^, n G N, then we say that 
f{x) is a worst case time upper hound for A (with respect to the size function 
size) if there exists a constant C G M such that for every w € M 

TaIw) Cf{size{w)) + C. 

The set 

Mj = {w G M I Ta{w) ^ f{size{w))} 

is called the f -stratum of A. 

Assume now that the set M comes equipped with a (finitely additive) mea- 
sure fi which takes values in [0, 1]. A subset Q C M is called generic {negligible) 
if /j{Q) = 1 (m(Q) — 0)- A. bound / is called a generic upper bound for A if 
the set Mf is generic with respect to /z. A generic upper bound / is tight if 
it is a minimal (with respect to the standard order ^ on the bounds) generic 
upper bound for A from a fixed list of upper bounds U. If not said otherwise, 
we always assume that U consists of polynomial bounds x" and exponential 
bounds G N. It may happen that an algorithm A does not have a tight 

generic upper bound. 
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If / is a tight generic upper bound for A then the stratum Mf is called a 
generic stratum. Sometimes it is difficult to determine generic strata precisely, 
in which case it is convenient to replace Mf by a "large enough" part of it. To 
this end we introduce the following notion. A subset RP C Dom(^) is called 
a Regular Part of M relative to A if RP is a generic subset of M such that 
RP C Mf for some tight generic upper bound / for A. One can view RP 
as the set of "algorithmically typical" inputs for A with respect to /i, so RP 
describes the most typical behavior of the algorithm on M. The compliment 
BH ~ M \ RP is called a black hole. Clearly, the regular part PR and the 
black hole BH are defined up to a negligible set. In applications BH consists 
of elements w in M for which either the algorithm A does not work at all, or 
T_a{w) is not bounded by f{size{w)), or for some reason it is just not known 
whether w is in Mf or not. Finally, for a bound h G U we say that the regular 
part RP of A has at most h time complexity if Mh is generic. In particular, we 
say that RP is polynomial time if it has at most h time complexity for some 
polynomial h. 

In the sequent the measure /i appears either as the asymptotic density func- 
tion on M with respect to the size function size, or the exponential distribution 
on M which comes from a corresponding random walk on M (we refer to [51 110| 
for details). To explain this we need a few definitions. Let 
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be a partition of M with respect to the given size function size : M ^ R, thus 

Mi ^ {w E AI \ size{w) — i}. 

In this case for a subset Q of M the fraction 

fi{Q n Si) 
KM,) 

can be viewed as the probability of an element of M of size i to be in Q. The 
limit (if it exists) 

p{Q) = hm 

l^OG fJ,[Mi) 

is called the asymptotic density of Q. The set Q is generic (negligible) with 
respect to p if p{Q) = 1 {p{Q) = 0), and Q is strongly generic if the convergence 

KSz) 

is exponentially fast when i — » oo. 

It is not hard to see that the union JF of all generic and negligible subsets of 
M is an algebra of subsets of AI and the asymptotic density p is a measure on 
the space (M, J^). 
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1.3 Search problems in groups 

The Word and Conjugacy Problems are two classical algorithmic problems in- 
troduced by M. Dehn in 1912. Since then much of the research in combinatorial 
group theory was related to these problems. We refer to surveys [TI I^I^IHS] 
on algorithmic problems in groups. 

Let H he a fixed group given by a finite presentation H — {X;R), and 
M{X) = a free monoid over the alphabet X^^. Sometimes, slightly 

abusing notations, we identify words in M{X) with their canonical images in 
the free group F{X). 

In general, an algorithmic problem over H can be described as a subset D 
of a Cartesian power M(X)'^ of M{X). The problem is decidable if there exists 
a decision algorithm A which on a given input w G M{X)^ halts and outputs 
"Yes" if w G £>, otherwise it outputs "No". Notice, that, on the first glance, 
decidability of the problem depends on the given presentation {X;R) of H. 
However, if the problems under consideration are "algebraic", i.e., concerning 
elements, subgroups, automorphisms, etc., then they are decidable relative to 
one finite presentation if and only if they are decidable relative to any finite 
presentation. 

Most of the algorithmic problems for groups come in three variations: spe- 
cific, uniform, and search. 

The classical formulations of the Word and Conjugacy Problems are specific, 
i.e., for a fixed presentation H = {X;R) find an algorithm (specific to this group 
presentation) to solve these problems in H. However, this particular meaning of 
an algorithmic problem has changed somewhat in recent years under influence 
of practical computations with groups. Development of software packages for 
computing in groups often requires implementation of uniform decision algo- 
rithms that are able to deal with a given algorithmic problem in various classes 
of presentations of groups. This might change the flavor and computational 
complexity of the problem dramatically. For example, the uniform Word Prob- 
lem for a class of presentations V has a pair: a presentation {X; R) € V and a 
word w e F(X) - as its input, and it requires to verify whether or not w is equal 
to 1 in the group H = {X;R). To see the difference, assume that one solves 
the (speciflc) Word Problem in a nilpotent group H = {X;R). Since H has a 
faithful matrix representation pH : H UTn{'I) for a suitable n one can replace 
the generators x £ X hy their images ph{x) and perform matrix multiplication 
to check if ph{w) = 1 or not. In the case of the uniform variation of the Word 
Problem the algorithm above would require first to find a faithful representation 
pp for a given finite presentation P £ V - not easy task in itself. Similarly, if 
the group H = {X;R) is hyperbolic then one can use the famous Dehn algo- 
rithm to solve the (specific) Word Problem in H. Namely, let P' = {X; R') be 
an arbitrary finite Dehn presentation of H (it always exists in this case). It is 
known that w = \ 'm H \i and only if the Dehn algorithm relative to P' rewrites 
the word w into the empty word. The described algorithm is very fast (linear 
time in the length of w, see [4]), but it relies on the knowledge of the Dehn 
Presentation P' . liV is the class of all finite presentations of hyperbolic groups 
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then the uniform version of the algorithm would require for a given presentation 
P G V to compute a Dehn presentation P' for the group H - which is again very 
demanding (there are no polynomial time algorithms known at the moment). 

In this paper we study uniform algorithms for the Word and Conjugacy 
Problems in the class of groups which are free products with amalgamation 
given by their standard presentations. 

Observe, that the uniformity of the problem may appear on different levels, 
not related to sets of presentation at all. For example, the specific Membership 
Problem in H is decidable for a given fixed finitely generated subgroup D of H 
if there exists an algorithm which for every word w € F(X) decides whether the 
element represented by w in belongs to D or not. Meanwhile, decidability 
of the uniform Membership Problem for H requires an algorithm which would 
solve the specific Membership Problem for every finitely generated subgroup D 
oiH. 

Finally, the search variation of an algorithmic problem D requires to decide 
whether a given w belongs to D or not, and if it belongs, to provide a "reasonable 
proof that w is, indeed, in D. For instance, the Search Word Problem for 
H = {X\ R) usually requires to check if w e F{X) is equal to one in _ff , and if 
so, represent w as a product of conjugates of relators from R. The Search Word 
Problem is sometimes provably harder then the solution of the original Word 
Problem. Indeed, the group BS{1, 2) = (a, 6; a^^ba = h"^) has a polynomial time 
decision algorithm for the Word Problem, but its Dehn function is exponential 
|19| . so it requires, in the worst case, at least exponential time to represent 
ui as a product of conjugates of the relator. This new requirement for the 
search decision problems to provide a "proof , or a "witness" , of the correct 
decision brings quite a few new algorithmic aspects, which were not studied in 
group theory. We refer to [211 133 for a more detailed discussion of the Search 
Problems in groups. 

Search problems could also be uniform or specific. It is convenient to treat 
uniform and specific forms as particular cases of problems which are uniform 
relative to a given class of objects More precisely, let D be an algorithmic 
problem on a set of inputs /. We say that D is decidable on a subset $ C / if 
there exists a partial algorithm A with a halting set Dom{A) C / that correctly 
solves D on every input from Dom{A) and such that $ C Dom{A). 

For example, the membership problem for a class of subgroups $ of solves 
the specific membership problem for every subgroup D from If the set $ is 
the whole set of elements, finitely generated subgroups, etc., of H then we omit it 
from the notation. This relative approach is very natural, since there are groups 
in which the uniform version of a particular algorithmic problem is undecidable, 
but still there are interesting subclasses of objects <& for which this problem is 
uniformly decidable. Moreover, even if the uniform version of the problem is 
decidable the class of all objects in the question can be partition into different 
subclasses with respect to different complexities of the decision algorithms. 

Below we list some algorithmic problems for H in their uniform relative 
to a subclass search variation. These algorithmic problems involve different 
subsets of H (subgroups, cosets, double cosets, regular sets, etc.) given by some 
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natural effective (constructive) descriptions. For example, finitely generated 
subgroups D are given by finite generating sets (which are given as words from 
F{X)), cosets wD are given as pairs {D,w), regular sets are given either by 
finite automata or by regular expressions, etc. Usually, we do not specify any 
particular descriptions of these subsets, unless it is required by complexity issues 
or by a particular algorithm. 

The Search Word Problem for finitely presented groups has several formu- 
lations which depend on the form of the witness. The following is, perhaps, the 
most typical one. 

The Word Seetrch Problem for a given subset of elements $ (WSP*): 

Let ^ be a given subset of elements from, H (given as words from F{X)). For 
a given w € ^ decide whether w = 1 in H or not? If w = 1 then find a 
presentation of w as a product of conjugates of relators from R. 

However, in free products with amalgamation it is convenient to consider the 
following variation of the Search Word problem. Let A/" be a fixed set of normal 
forms (viewed as words in M{X)) of elements from G, and w a representative 
of tu in TV. 

The Normal Forms Search Problem for a given subset of elements $ 
(NFSP#) of H: Let ^ be a given subset of elements from H (given as words 
from F{X) ). For a given w G ^ find its normal form w G M. 

In free products with amalgamation and HNN extensions it is convenient to 
begin with the reduced forms of elements and then specify the normal forms and 
cyclically reduced normal forms among them. One can introduce similarly the 
Search Problems for reduced and cyclically reduced normal forms. We leave it 

to the reader. 

The Conjugacy Search Problem for a given set of pairs of elements $ 
(CSP$): Let ^ be a given set of pairs of elements from H. For a given pair 
(u, v) e ^ determine whether u is a conjugate ofv in H or not, and if it is then 
find a conjugator. 

The Membership Search Problem for a set of subgroups $ (MSP*) 

of H: Let ^ be a set of finitely generated subgroups of H. For a given D e $ 
and a given w £ F(X) determine whether w belongs to D or not, and if so, find 
a decomposition of w as a product of the given generators of D. 

The Conjugacy Membership Search Problem for a set of subgroups 
$ (CMSP$): Let ^ be a set of finitely generated subgroups of H . For a given 
D G $ and a given w € F{X) determine whether w is a conjugate of an element 
from D, and if so, find such an element in D and a conjugator. 

The Coset Representative Search Problem for a set of subgroups $ 
(CRSP#): Let ^ be a set of finitely generated subgroups of H . For a given 
D G ^ find a recursive set S of representatives of D in H and an algorithm As 
which for a given word w G F{X) finds a representative for Dw in S 

Observe that to solve CRSP# for a given D G $ it suffices to find the 
algorithm As, since w G S ii and only if w is the output of As on the input w. 
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To formulate the next algorithmic problem we need the following definition. 
Let M be a subset of a group H. If u,v € H then the set uMv is called a 
shift of M. For a set M of subgroups of H denote by ^{M,H) the least set of 
subsets of H which contains Ai and is closed under shifts and intersections. 

The Cardinality Search Problem for <^{M,H) (CardSP^.): Let M be 

a collection of subsets of H . Given a set D G ^{M,H) decide whether D is 
empty, finite, or infinite and, if D is finite non-empty, list all elements of D. 

1.4 Results 

We show below that, under some reasonable assumptions about the groups in 
the free product with amalgamation G = A *c B, the Normal Forms Search 
Problem for the classical normal forms in G is decidable and the Search Con- 
jugacy Problem is decidable for the set of regular elements in G. Moreover, 
we analyze the time complexity of these problems (modulo the corresponding 
algorithms in the factors A, B). 

In Section 13.21 we study time complexity of the standard Algorithm II for 
computing the normal forms of elements in amalgamated free products (as, 
basically, described in [27 ). 

The direct inspection of the algorithms reveals two key issues that determine 
its time complexity. The first one is related to the time complexity of the deci- 
sion algorithms of the relevant algorithmic problems that have to be decidable 
in the factors (the Subgroup Membership Search Problem and the Subgroup 
Representatives Search Problem) . The second issue concerns with the following 
phenomena that occurs when computing the required form (normal or reduced) 
for the input w = <7i52 ■ • ■ ffn, Qi G AVJB in the group A*c B. While in progress, 
Algorithm I gradually rewrites the words in generators of A and B representing 
the elements gi, possibly increasing their length. It happens sometimes that the 
accumulated increase in length is exponential in terms of the original length. 

These two issues are very different in nature, the algorithmic difficulties of 
the first type come with the factors and we view them as part of the given data. 
Meanwhile, the ones of the second type are intrinsic to the construction itself. 
To deal with the first issue we elected to specify precisely which basic algorithmic 
problems are required to be decidable in the factors and study algorithms for 
A*c B "modulo" the time complexity of the basic problems in the factors A, B. 
In this case one can view every instance of execution of a basic algorithm (a 
decision algorithm corresponding to a basic problem) as one " elementary step" . 
It turns out that if the complexity of the basic algorithms, as well as the intrinsic 
complexity, is known then one can estimate the total complexity of Algorithms 
I. 

As an example of this kind of analysis we give the following result for free 
products of free groups in Section [3?3l 
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Theorem 3.7 

(1) Let A *c B he a free product of free groups with finitely generated amal- 
gamated subgroup C . Then Algorithm II has at most exponential time 
complexity function (in the syllable length of the input words); 

(2) There are finitely generated free groups A and B and a finitely generated 
subgroup C in A and B for which the lower (and upper) bounds on the 
time complexity of Algorithm II are exponential. 

In Section [5] we study the time complexity of the Search Conjugacy Prob- 
lem in G = A^c B. More precisely, we study the time complexity of the search 
variation of the standard decision algorithm for the Conjugacy Problem in amal- 
gamated free products (following the description in [:2T). The main result of 
this paper shows that this algorithm solves the Search Conjugacy Problem in 
G for all cyclically reduced regular elements, and their conjugates, provided the 
relevant basic algorithmic problems in the factors are decidable. We would like 
to emphasize that the algorithm is partial, i.e., it halts and gives the answer (al- 
ways the correct one) only for inputs from a subset of G (though a very big one). 
This is a crucial aspect of our approach, since, as we have mentioned above, the 
Conjugacy Problem in some of these groups is undecidable, hence there are no 
total decision algorithms for the Conjugacy Problem in these groups. We show 
in the subsequent papers that this partial algorithm is, perhaps, as good as a 
total one, and may be even better on the most typical inputs (if the total al- 
gorithm becomes very inefficient when trying to accommodate all non-regular 
inputs). 

To describe the regular elements in G we need the following definitions. The 

set 

N*aiC) = {ge\GnC^ ^1} 
is called the generalized normalizer of C in G [6 . Its " dual" is defined by 

Zg{C)= U c^'nc 

Now in the group G = A *c B the set 

BH = {N*g{G) \ C) U Zg{G) 

is called a Black Hole, and its complement RP — G \ BH is called the Regular 
Part of A, meanwhile the elements from RP are called regular. 

The following result shows that under reasonable assumptions on the factors 
the Membership Problem for the set of regular elements is decidable in G. 

Theorem 4.1 Let G ~ A^c B be a free product of finitely presented groups 
A and B amalgamated over a finitely generated subgroup G . Assume also that 
A and B allow algorithms for solving the following problems: 

• The Search Membership Problem for the subgroup C . 
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• Coset Representative Search Problem for the subgroup C . 

• Cardinality Search Problem for ^{Sub{C), A) in A and for ^{Sub{C), B) 
in B. 

• The Membership Problem for NXiC) , A^b(C) and Za{C), Zb{C). 

Then there exists an algorithm to determine whether a given element in G is 
regular or not. 

Theorem 4.18 Let G = A^c B be a free product of finitely presented groups 
A and B amalgamated over a finitely generated subgroup G . Assume also that 
A and B allow algorithms for solving the following problems: 

• The Membership Search Problem for the subgroup G . 

• Coset Representative Search Problem for the subgroup G . 

• Cardinality Search Problem for Sub{G) (see Section ] 

• Conjugacy Search Problem. 

• Conjugacy Membership Search Problem for G . 

Then the Conjugacy Search Problem in G is decidable for cyclically reduced 
regular elements from G and their conjugates. 

Corollary 4.19 Let G — A^cB be a free product of free groups A and B with 
amalgamated finitely generated subgroup G . Then the Conjugacy Search Problem 
in G is decidable for cyclically reduced regular elements and their conjugates. 

It is worthwhile to mention that the conjugacy problem for elements of the 
syllable length greater than 1 is somewhat easier (it requires less conditions on 
the factors). 

Theorem 4.15 Let G — A*c B be a free product of finitely presented groups 
A and B amalgamated over a finitely generated subgroup G. Assume also that 
A and B allow algorithms for solving the following problems: 

• The Membership Search Problem for the subgroup G . 

• Coset Representative Search Problem for the subgroup G. 

• Cardinality Search Problem for Sub{G). 

Then the Conjugacy Search Problem in G is decidable for cyclically reduced 
regular elements g of syllable length l{g) > 1. 

At the end of this section we briefly discuss some connections with the sub- 
sequent papers of the series. 

In the paper we give asymptotic estimates of the size of the regular part 
RP and the black hole BH in free products of free groups with amalgamation. 
This enables us to show that Algorithm II, as well as the algorithm that solves 
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the Search Conjugacy Problem in such groups, both have a polynomial generic 
case time complexity. 

In the paper |12| the main results on the normal forms and the Conjugacy 
Search Problem in the free products with amalgamation are generalized to HNN 
extensions. To this end we introduce the so called transfer machines which allow 
one to "transfer" effectively the results on the classical algorithmic problems 
for free products with amalgamation to the corresponding HNN extensions of 
groups. 

In the paper [13j we apply the general results obtained in the papers [111 
\T2\ to the Miller's groups which are particular types of HNN extensions. In 
particular, we show that, despite the conjugacy problem is undecidable in these 
groups, there exists an algorithm that solves the Conjugacy Search Problem in 
the Miller's groups in polynomial time on "most inputs". 

2 Preliminaries 

2.1 Amalgamated products 

In this section we briefly discuss definitions and some known facts about free 
products with amalgamation. We refer to [57] for details. 

Let A, B, C be groups and (f) : C — > A and ip : C — > B be monomor- 
phisms. Then one can define a group 

G = A*cB, 

called the amalgamated product of A and B over C (the monomorphisms 0, ip 
are suppressed from notation). If A and B are given by presentations 

A^{X\Ra^I), B^{Y\Rb^I), 

and a generating set Z is given for the group C, then the group G has presen- 
tation 

G (X u r I i?A = 1, i?B = 1, = z'^iz e z)). (1) 

Notice that if the groups A and B are finitely presented and C is finitely gener- 
ated then the group G is finitely presented, li Z = {zi, . . . , z„} and we denote 

zf^u^iX), zf^v.iY) 

then G has presentation 

G = (X U y I i?A = 1, -Rs = 1,1*1 = Ui, . . . , u„ = Vn). 

The groups A and B a called factors of the amalgamated product G = A*cB, 
they are isomorphic to the subgroups in G generated respectively by X and Y. 
We identify A and B with these subgroups via the identical maps x ^ x, y ^ y 
{x e X, ye Y). If we put 

V = {ui,...,Un}, V ^{vi,...,Vn} 

then 

G ^ (U) = {V) = AdB i^G. 
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2.2 Normal forms of elements 

Let G ~ A*c B he an amalgamated product of groups as in Denote by S 
and T fixed systems of right coset representatives of C in A and B. Throughout 
this paper we assume that the representative of C is the identity element 1. 

The following notation will be in use throughout the paper. For an clement 
g G {AU B) \C we define F{g) ^ A if g E A and F{g) ^ Biig eB. 

Theorem 2.1. [571 Theorem 4.1] An arbitrary element g in G — A *c' B can 

be uniquely written in the normal form with respect to S and T 

g = cgig2---gn, (2) 
where cGC,giGTUS \ {1}, and F{gi) ^ F{gi^i), i ~ 1, . . . ,n, n ^ 0. 
Corollary 2.2. Every element g G A *c B can be written in a reduced form 

g = cgig2 ■ ■ ■ gn (3) 

where c £ G,gi G (A U B) \ C, and F{gi) ^ F{gi+i), i — 1, . . . ,n, n ^ 0. 
This form may not be unique, but the number n is uniquely determined by g. 
Moreover, g ^ 1 if and only if n = and c — I. 

Let g G A *c B and g — cgig2 ■ ■ ■ gn be a reduced form of g. Then the 
number n is called the length of g and it is denoted by l{g). Observe, that 
l{g) = Q^gGC. 

Definition 2.3. Let g G A *c B. A reduced form g = cgig2 ■ ■ ■ gn is called 
cyclically reduced if one of the following conditions is satisfied: 

(a) n = 0; 

(b) n — 1 and g is not a conjugate of an element in C; 

(c) n^2 and F{gi) ^ F{gn). 

Notice that our definition of cyclically reduced forms is slightly different 
from the standard one (see, for example, [27]). Usually, the condition (b) is not 
required, but the difference is purely technical, and it is convenient to have (b) 
when dealing with conjugacy problems. Observe also, that if one of the reduced 
forms of g is cyclically reduced then all of them are cyclically reduced. In this 
event, g is called cyclically reduced element. 

Lemma 2.4 (|27p. Let g G A *c B . Then g is a conjugate of some element 
go in a cyclically reduced normal form. This element go not uniquely defined, 
but its length l{go) is uniquely determined by g. 

The normal form of go is called a cyclically reduced normal form of g. The 
uniquely determined number l{go) is called the cyclic length of g and it is denoted 
by ^0(5)- Observe that 

^0(5) = <^==> some conjugate of g is in C, 

^0(5) = 1 ■^^==> some conjugate of g is in {AU B) \ G. 
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2.3 The conjugacy criterion 

Theorem 2.5. 127\ Theorem 4-S] Let G — A*c B be an amalgamated product, 
and let g be a cyclically reduced element in G. 

(i) If lo{g) = 0, i.e., g d C, and g is conjugate to an element c & C then 
there exists a sequence of elements c = cq, ci, . . . , ct = g, where Ci € G 
and adjacent elements Ci and Q+i, i — 0, . . . ,t — 1, are conjugate in A or 
in B. 

(ii) If lo{g) = 1, i.e., g G AU B \ C , and g' is a cyclically reduced element 
which is a conjugate of g in G then l{g') = 1, Fig) = Fig') cind g and g' 
are conjugate in Fig). 

(iii) Let ^0(5) — r ^ 2 and g = gi ■ ■ ■ gr he a, cyclically reduced form of g. 
Assume that g is a conjugate of a cyclically reduced element h — hi ■ ■ ■ hs 
in G. Then r — s and h can be obtained from g hy a cyclic permutation 
of the elements 51, ... ,5,- followed by a conjugation by an element from 
G. 



2.4 Malnormal subgroups 

Recall, that a subgroup H oi & group G is called malnormal in G if fl = 1 
for all 5 e G \ 

It follows immediately from the conjugacy criterion ( Theorem l2.5p that free 
factors A and B are malnormal in the free product A * B. It is known that 
maximal abelian subgroups (= proper centralizers) are malnormal in torsion- 
free hyperbolic groups, in particular in free groups. We refer to [50] for results 
on malnormality of maximal abelian groups in free products with amalgamation 
and HNN extensions. 

Definition 2.6. Let G be a group and H he a, subgroup of G. The generalized 
normalizer NQiH) is a set of all elements g G G such that H n 1. 

Notice that, NciH) C NQiH), and, in general, Nq{H) is not a subgroup. 
It is obvious that if 5 G Nq{H) then N^iH) contains the whole double coset 
HgH. A set of representatives {gi | « £ /} of double cosets of H is called a 
double transversal of H in N^iH), in this event 

N^iH)^\jHg,H 

If is a finitely generated subgroup of a free group G then H has a finite 
double transversal in NQiH), moreover such a transversal can be found algo- 
rithmically [5] . A more convenient algorithm (in terms of subgroup graphs) can 
be found in ^5] . 

For an element 5 G G define 

Zg{H) = {heH \ hs eH} = H^'' n H 
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and put 

Zg{H) = U Z,{H). 

Even though Zg{H) is a subgroup of G for every g £ G, the set Zg{H) may not 
be a subgroup. Observe, that for any u,v € H 

Zugv{H) = Zg^ {H). 

Hence if T is a double transversal of H in Nq{H) then Zg{H) is union of 
conjugacy classes: 

ZciH) = y Z,iHf- 

heH,teT,t=ii 

In particular, if the transversal T is finite then Zg{H) is union of finitely many 
conjugacy classes of subgroups Zt{H). 

Definition 2.7. Let G be a group equipped with a map L : G — » N and H be 

a subgroup of G. For an element g <E G define Lnig) as the minimal value of 
L on the double coset HgH. Then the malnormality degree md{H) of in G 
with respect to L is the smallest number r such that H n = 1 for all 5 e G 
with Lnig) ^ r, if it exists, and 00 otherwise. 

For example, the malnormality degree of subgroups can be defined in free 
groups, free products with amalgamation, and HNN extensions of groups with 
respect to the canonical length functions. In the sequel we always assume that 
for _ff ^ A*cB the degree md{H) is viewed with respect to the canonical length 
function I : A*c B —>^. 

Obviously, if a subgroup H has a finite double transversal in Nq{H) then 
md{H) is finite. 

Lemma 2.8. Let G = A*c B and D ^C. Then 

(i) If C is malnormal in A and B then md{D) — 1. 

(ii) If C is malnormal in one of the groups A and B then md{D) < 2. 

Proof. Let g = 51 be a reduced form of an element g € G. Suppose 

l{g) ^ 1, in particular, gn ^ C. Suppose also that c, c' € G. If 

91 ■ ■ ■ 9ncg~^ ■ ■ ■ gx'^ =c' 

then gncg^^ € C. Assume that G is malnormal in both A and B. This implies 
that gn & C -contradiction. Then n = and therefore md{D) = 1. 

If G is malnormal either in A or in B then similar argument shows that 
md{D) < 2. □ 

Question 2.9. Let G = A *c B and be a finitely generated subgroup of 
G. Assume that the malnormality degree mdaiC) of G in G is finite, and H 
contains no elements of length ^ mdaiC). 

(a) Is it true that md(II) is finite? 

(b) Is it true that Nq{H) is union of finitely many double cosets of HI 
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3 Computing normal forms 

3.1 Computing reduced forms: Algorithm I 

In this Section we discuss the standard algorithm to compute reduced forms of 
elements of a group G = A*c B. Suppose that the Membership Search Problem 
MSP for C is decidable in A and B. 

Observe, that given a word g £ F{X U Y) one can effectively present it as a 
product 

5 = 3i---5fc, (4) 
where gi, . . . , gk are reduced words in X or in 1" and if gi is a word in X then 
is a word in Y and vice versa. 

Algorithm I: Computing Reduced Forms. 
Input: a product g ~ gi ■ ■ ■ gk in the form 
Step 1. 

Check ii gi G C , i = 1, . . . , k or not. If none of the t^^'s lies in C then ([4]) 
is reduced. 

Step 2. 

Otherwise, we look at the first on the left word gi such that gi ^ C and 
transform the word g according to one of the rules: 

• If G then rewrite gi into a product Ci(ui, . . . , m„) of the 
given generators of C"^, replace gi by Ci(wi, . . . , using substi- 
tution Uj —^ Vj,j = l,...,n, and then replace g = gi ■ ■ ■ gk by 
91 ■ ■■9t-2g[gt+2 ■■■gk, where g[ = g,^ic,{vi, . . .,Vn)gt+i; 

• If gi e F{Y) then rewrite gi into a product Ci(wi, . . . , d„) of the 
given generators of , replace gi by Ci(ui, . . . , u„), using substi- 
tution Vj — > Uj,j — l,...,rt, and then replace g — gi ■ ■ ■ gk by 
91 ■ ■■9t-29lgt+2 ■■■gk, where g- = . . . ,u„)gj+i; 

thus decreasing the syllable length l{g) of g. Go to Step 1. 
Output: The word 

g = Ui - ■ ■Urn 

which is a reduced form of g. 

End of Algorithm II 

Notice that to carry out this algorithm one needs to be able to verify whether 
or not a given word gi £ F{X) {gi G F{Y)) belongs to the subgroup C"^ in A 
{C^ G B), and, if so, then to rewrite gi as a word in the given generators U 
of C"^ {V of C"^). Hence, the Search Membership Problem SMP has to be 
decidable for the subgroup C in A and B. 

Lemma 3.1. |27j Let G = A *c B and MSP is decidable for G in A and in 

B then Algorithm I finds a reduced form of elements of G. 
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3.2 Computing normal forms: Algorithm II 

In this section we discuss the fohowing algorithmic problem. 

Normal Forms Search Problem: Let G = A*c' B and let S,T be recursive 
sets of representatives of A and B modulo C . Give an algorithm which for a 
given g G F{X U Y) finds the normal form of g in G with respect to the sets S 
and T. 

Now we describe the standard known decision algorithm for the problem above 
(see, for example, [21]) provided we are given decision algorithms for the Mem- 
bership Search Problem MSP and the Coset Representatives Search Problem 
CRSP for the subgroup C in A and in B (relative to the sets S and T). 
Given a word g G F{X U Y) one can effectively present it as a product 

a = gi---9k, (5) 

where gi, . . . ,gk are reduced words in X or in Y , and if gi is a word in X, then 
gi+i is a word in and vice versa. 

Modulo decision algorithms for problems MSP and CRSP for the subgroup 
C in the groups A and B the process of computing the normal form is the 
following. 

Algorithm II: Computing Normal Forms. 
Input: a word g ~ gi- ■ ■ gu in the form ([5]). 
Step 1. 

(a) If gk is a word in X, then: 

(a.l) Write it as gk — CkUk with Uk G S, Ck 6 C given as a word 
Ck{ui, . . . , Un) in the given generators U of C"^ (using MSP and 
CRSP). 

(a. 2) Then rewrite Cfc(iti, . . . ,u„) into Cfc(ui, . . . ,w„). 

(a.3) If Ufe 7^ 1 then replace g = gi ■ ■ ■ gk hy g = gi ■ ■ ■ gk-2g'k-iUk, 
where g'^._i = {gk-iCkivi, . . . , «„)). Go to Step 2. 

(a.4) Otherwise replace g = gi ■ ■ ■ gk hy g = gi ■ ■ ■ gk-2g'k-i- Go to 
Step 1. 

(b) If gk is a word in Y then: 

(b.l) Write it as gk = CkUk with Uk € T Ck & C given as a word 
Cfc(ui, . . . , w„) in the fixed generators V of (using MSP and 
CRSP). 

(b.2) Then rewrite Ck{vi, . . . ,ii„) into Cfc(wi, . . . ,««)■ 

(b.3) If Ufc 7^ 1 then replace g ^ gi ■ ■ ■ gk hy g = gi ■ ■ ■ gk^2g'k-iUk, 

where g'^_^ = {gk-iCk{ui, . . . , u„)). Go to Step 2. 
(b.4) If Uk = 1 then replace g = gi ■ ■ ■ gk hy g ^ gi ■ ■ ■ gk~2g'k-i- Go 

to Step 1. 
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Step 2. If 5 is represented in the form (O: 

9 = 9i- --gtUi+i ■ ■ -Ura, 
where Wi+i • • • is in the normal form, DO; 

(a) Execute Step 1 on gi ■■■ gi. 

(b) If the outcome of the Step 1 on gi ■ ■ ■ gi is 51 • • ■ g[_i, i.e., Ui = 1, 
then replace g ^ gi ■ ■ ■ gtUi+i • • • by 5 = gi • • • g-.^u^+i • • • u,„ and 
go to Step 2. 

(c) If the outcome of the Step I on gi ■■■ gi is gi ■■ ■ g[_-^Ui, i.e., Ui ^ I, 
then: 

(c.l) If both Ui and iti+i are in S (or both of them are in T), rewrite 

UiUi+i into c'u[ where c' G C, given as a word c'(wi, . . . , «„) (or 

c'(ui, . . . , and w- G S* (or G T). 
(c.2) If ^ 1 then replace g ^ gi ■ ■ ■ giUi+i ■ ■ ■ Um by g = 51 • • • g'/_j^u'^Ui+i ■■ - u, 

where g'/_i — g'i^ic' . Go to Step 2. 
(c.3) If M- = 1 then replace g = gi ■ ■ ■ giUi+i • • • m,„ by g = 51 • • • g'/^^^Ui+i ■ --Um, 

where g'/_i = g-_ic'. Go to Step 2. 

Output: The word 

g = CiUi ■ ■ - Um 

which is the normal form of g relative to the set of representatives S and T. 

End of Algorithm II 

We summarize the discussion above as the following theorem 

Theorem 3.2. Let G = A*c B and the problems MSP and CRSP are de- 

cidable in A and in B for the subgroup C. Then Algorithm II finds the normal 
form of g for every given element g E G. 

3.3 Complexity of Algorithm II 

Now we discuss briefly time-complexity of Algorithm II. Recall that the time 
function T4 of an algorithm A is defined on an input g of A as the number of 
steps required by the algorithm A to halt on the input g. 

Obviously, the complexity of the time function Tj of the Algorithm II de- 
pends on complexity of the time functions of decision algorithms for MSP and 
CRSP for C relative to A and B. Also, it depends on how the length of the 
words Ci grows during the execution of Algorithm II. 

Complexity of MSP and CRSP depends on particular groups A, B, and 
C. For example, if A and B are free groups, then these problems have linear 
time complexity for a fixed subgroup C (see, for example, [25]). 

Estimating the complexity of the rewriting process (a) is more demanding, 
even in the case of amalgamated products of free groups. Recall, that in the 
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rewriting process (a), executing the instructions (a. 2) or (b.2), we rewrite a word 
Cj+i(iti, . . . , Un) into a word cj+iivi, . . . , Vn)- Set 

ma.x{\ui\,...,\un\} 
mm{|wi|, .... \vn\\ 

Then we have an upper bound estimate on the increase of the length 

\Cj+i{vi,...,Vn)\ < X{u,v) ■ |Cj+i(ui,...,«„)|. 

Similarly, in the case when we rewrite a word cj+i given in the generators Vi into 
a word in generators Ui we have an estimate with the factor X{v, u). Therefore, 
if we denote 

A = max{A('U, v), X{v, u)} 

then at any rewriting step one has increase in length of at most by the factor A. 

Now suppose, for simplicity, that the length of Cj increases in executing all 
other instructions, different from for (a. 2) and (b.2), at most by M+ \gj\ where 
M is a fixed constant (we make this assumption to focus on the processes (a.2) 
and (b.2)). Under these assumptions 

|c,K A- |c,+i|+M+|ff,-| (6) 

In particular, if the length of Cj does not increase at all in the rewriting processes 
other than (a.2), (b.2), then in k steps we will have an exponential estimate 

|ciKA'=-i-|cfc| 

where k = l{g). So if A > 1 then we might have exponential growth of the length 
of the words Cj. The example below shows that this happens in the worst case 
scenario. 

Example 3.3. Let A = F{a, b,d),B = F{a, b, d) be two free groups of ranks 3. 
Consider two subgroups of rank 2: 

C={aP,b) ^A,C={dy) s^B, 

where p ^ 2 is an integer. Then the map (f> defined by ^(a'^) = a,(j){b) = b'' 
gives rise to an isomorphism (j) : C —>■ C. Put 

G = A *c=c ^ = ^' d,d,b,d\aP = d,b = 6^). 

Let 5 be a recursive set of representatives of A modulo C such that the repre- 
sentative in S of the coset Cda^"^ is b^P"^daP"^ for all integers m. In particular, 

daP"" = bP"'{b-P"'daP"') (m e Z) 

It is not difficult to construct such S since the set of elements of the type 
rfa^"* is recursive, as well as cosets of C. Similarly, let T be a recursive set of 
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representatives of B modulo C such that the representative in T of the coset 
(Jdlfpm jg --pm^j^pm £qj, integers m, which implies that 

Now consider the following element in G: 

g = dddd ■ ■ ■ ddd = gi - ■■ gu 

Then, in the notations of Algorithm II, the rewriting processes (a. 2) and (b.2) 
go as follows: 

Ck =a = 

gk-i = d,gk-iCk = da" = }f{h~PdaF) = Wuk-i 
Now the next step will be 

Cfe_i =hP = ,gk-2 = d 

Hence 

— ~ 2 2 2 — 2 2 

gk-2Ck-i = dlf = aP (a~P dU' ) ^ aF ■ Uk-2 = Ck-2 ■ Mfc-2- 

In this case X = p, lengths of the words Ci do not change in the rewriting 
processes other then (a. 2), (b.2), so the word Ci grows every step by a factor of 
p, hence 

|ci|=/ 

where k = l{g) — 1. 

Example 3.4. Let A = F(a,b), B = F{a',b') be two free groups of rank 2. 
Consider two subgroups of rank 2, C = (a, a*"), C — (^a'^, a"' ^, where p ^ 2 is 

an integer. Then the map <j) defined by 0(a) = a'^ and (j){a'') = a'" gives rise to 
an isomorphism (j)C C. Put 

G = A *c=C' B = (a, b, a', b'\a = a"'' , a'' = a'"^ 

Let 5* be a recursive set of representatives of A modulo B such that every element 
from (6) is in S. Analogously, T is the set of representatives B modulo C" such 
that every element from (6') is in T. Now consider the following element in G: 

g = {bb'^aibbT = a^" 

Rewriting of this element into the normal form involves the exponential growth 
of the lengths of intermediate words Cj. 
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Now we turn to the complexity of rewriting processes in Algorithm II other 
than (a. 2), (b.2). In general, this complexity depends on the particular algo- 
rithms for solving MSP and CRSP for C in A and B. In the case of free 
groups A and B the decision algorithm in [5S] for solving MSP and CRSP 
have some important features. If we denote by w the representative of the 
coset Cw produced by the algorithm on the input word w, then the following 
conditions hold: 

• For a given w E A the representative w of the coset Cw has the minimal 
possible length in Cw. 

• There exists a constant M such that for a given w E A if w = cw for a 
(unique) c £ C then |c| ^ \w\ + M. 

• the time spent by the algorithm on an input w is bounded from above by 
L\w\ for some fixed constant L. 

This allows one to estimate the complexity of Algorithm II in the case of 
free groups. From now on we assume that Algorithm II has subalgorithms for 
solving MSP and CRSP which satisfy the conditions above. 

Lemma 3.5. Let A^c B he a free product of free groups with finitely gener- 
ated amalgamated subgroup C . Then the lengths of the words Ci that occur in 
computations with Algorithm II on an input w is bounded from above by 



where k = l{w). 



Proof. Let w — gi . . . g^ he an input for Algorithm II in the form ([5]) , where 
k = l{w). It requires k steps for Algorithm II to produce the input. According 
to (O on each step the length of the word Cj is bounded by 

\cj \ < A • \cj+i\ +M+\gj\^X- |cj+i| + M + \w\. 

Hence in k steps we will have the following estimate on the lengths of the words 
Cj , j — 1, . . . k. 

A(---(A(|u'|+A/) + |u;|+M))---) + |w|+A/ = (A^^-^ + • • ■ + l)(|u;| + M) 

as required. □ 

Corollary 3.6. Let A*cB be a free product of free groups with finitely generated 
amalgamated subgroup C . Then the time spent by Algorithm II on an input w 
is bounded above by 

k-Li-lwl-X'' -ilwl+M) 
where Li is a fixed constant and k — l{w). 
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Proof. Indeed, Algorithm II works k steps on an input w with l{w) — k. On 
each step it rewrites a current word Cj of the length bounded from above in ([7]) . 
The rewriting involves the subalgorithms for solving MSP and CRSP. These 
algorithms spend at most linear time with respect to the length of the input. 
Putting all the estimates together we have the resulting estimate above. □ 

Combining the corollaries above with the example we have the following 
result. 

Theorem 3.7. 

(1) Let A *c B be a free product of free groups with finitely generated amal- 
gamated subgroup C . Then Algorithm II has at most exponential (in the 
length of the input words ) time complexity function bounded by: 

kLi\w\\^ {\w\ + M) 

where fc,Li,A, Af, and w are as above; 

(2) There are finitely generated free groups A and B and a finitely generated 
subgroup C in A and B such that in the free product with amalgamation 
A*c B the Algorithm II has precisely the exponential time complexity as 
above. 

However, we will show in the subsequent paper [TT] that the situation in 
the example above is very rare, and in every free product with amalgamation 
G = A*G B of free groups with a finitely generated group C the Algorithm II 
is very fast on generic inputs. 

3.4 Computing cyclically reduced normal forms: Algo- 
rithm III 

In this section, we shall discuss the standard algorithm to find a cyclically 
reduced normal form of an element g of a group G — A *c B. As before, we 
assume that the element g is given in the form ([5]): 

5 = .91 • --dn, 

where gi, . . . , g„ e F{X) U F{Y) and g, e F{X) if and only if g,+i e F{Y). 

We work under assumption that the Membership Search Problem MSP, the 
Coset Representative Search Problem CRSP, and the Conjugacy Membership 
Search Problem CMSP are decidable in A and B for the subgroup C, and we 
have the decision algorithms in our possession. Notice that we need CMSP 
only because we have a slightly stronger notion of reduced forms than the usual 
one (see Section [221) • 

Observe, that the uniform version of CMSP is decidable in free groups and 
the decision algorithm has linear time complexity (in the length of the input 
word w) for a given finitely generated subgroup C |25j . 
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Algorithm III: Computing Cyclically Reduced Forms. 
Input: a word g in the form ([5]). 

Step 1 Find the normal form of g using the Algorithm II: 

9 = Cg5i ■ • - fffc- 

Observe that l{g) — k and for every gi we know its factor F{gi). 
Step 2 

(a) If l{g) = then g is already in cyclically reduced from. 

(b) If ^(5) = 1, for example, ii g ^ A, then check whether g is a conjugate 
of an element c £ C or not, using the algorithm for CMSP. In the 
former case, c is a cyclically reduced form of g and the algorithm for 
CMSP gives one of such elements c. In the latter case, g is already 
in cyclically reduced form. 

(c) Let l{g) ^ 2. 

— If F{gi) ^ F{gk), then g is already in a cyclically reduced normal 
form. 

— If F{gi) — F{gk). Then g is conjugate to 

Now apply the decision algorithms for MSP and CRSP to the 
word (gkCqgi) to find the normal form c'g'i of it. If g'l ^ 1 then 

c'g'i52 • ■■9k~\ 

is a cyclically reduced normal form of g. Otherwise, 

F(<^g2)^F(^gk-i) 

and we apply the procedure above to d g2 ■ ■ ■ gk-i- 

End of Algorithm III 

Lemma 3.8. Let G = A*cB and the problems MSP and CRSP are decidable 
in A and B for the subgroup C . Then there exists an algorithm that for a given 
element g £ G finds an element g' ^ G such that g' is a conjugate of g and if 
l{g') > 1 then g' is a cyclically reduced normal form of g. 

Proof. Direct analysis of Algorithm III shows that a decision algorithm for the 
problem CMSP is used only when executing instructions in the case (b). How- 
ever, if we modify Algorithm III in such a manner that it stops immediately when 
the case (b) occurs, then the modified algorithm satisfies the requirements of 
the lemma. □ 
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Theorem 3.9. Let G ^ A*cB and the problems MSP, CRSP, and CMSP 

are decidable in A and B for the subgroup C . Then for a given element g & G 
Algorithm III finds a cyclically reduced normal form of g in time Tiii{g) which 
can be bounded from above as follows: 

Tiiiig) s$ Tii{g)+K-max{TcMSp{cggi), {TMSp{9kCggi) + TcRSp{gkCggi))-l{g)}, 

where Tjj, Tsmp, Tcmsp, Tcrsp a'^e the time functions, correspondingly, of 
Algorithm II, and the decision algorithms for MSP, CMSP, CRSP, K is a 
constant. In particular, if A and B are free groups then 

Tiii{g)^Tiiig) + K,-\g\-l{g), 

where Ki is a constant (depending on C) and \g\ is the length of the input g 
given as a word in F{X U Y). 



4 Regular Elements and black holes 

4.1 Bad pairs 

LetG^A *c B. 

Definition 4.1. We say that (c, G C x G is a bad pair if c 7^ l, g ^ C, and 
gcg-^ e C. 

Notice that if (c, g) is a bad pair then g e Nq{C) \ C and c e Zg{C). The 
following lemma gives a more detailed description of bad pairs. 

Lemma 4.2. Let c G C \ {1} and g £ G \ C . If g = CgPi • • - pk is the normal 
form of g then (c, g) is a bad pair if and only if the following system Bc^g has a 
solution Cl, . . . , Cfc with Ci G C: 

PkCpl^ = Cl 
Pk-lClPl\ = C2 

PlCk-lPl^ = Ck 

Moreover, in this case pi G Np(p.){C) and c G Za{G) U Zb{G). 

Proof. This lemma is a particular case of Lemma 14.31 □ 

Observe, that consistency of the system Bc^g does not depend on a particular 
choice of representatives of A and B modulo C. Sometimes we shall treat c as a 
variable, in which case the system will be denoted _Bg. If c, ci, . . . , c^+i G C\{1} 
is a solution of Bg then we call it a nontrivial solution of Bg. 

Now we will study slightly more general equations of the type go = c' g' and 
their solutions c, c' in C . 
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Lemma 4.3. Let G ~ A *c B , g, g' E G be elements given by their normal 
forms: 

9^CgPi---pk, g' = Cg,v\- ■ -v'^ (fc ^ 1). (8) 

Then the equation gc — c'g' has a solution c,c'eC if and only if the following 
system Sg^gi in variables c, c', Ci, . . . , Cfe has a solution in C : 

PkC = cxv'k 

Pk-lCl = C2Pfe_i 
PlCfc-l = Ckp'i 

Proof. Let c, c' e C be a solution to the equation gc = c'g' . We then rewrite 
the equahty gc = c'g' as 

CgPi ■ ■ -PkC = c'cg'p'i ■■■p'k. 

Notice that the right hand side of this equahty is in the normal form. Following 
Algorithm II we shall rewrite the left hand side of this equality into the normal 
form. After rewriting the both sides must coincide as the normal forms of the 
same element. This gives rise to the system of equations for some elements 
c, c', ci, . . . , Cfc £ C, as above. Conversely, if the system Sg^g' has a solution then 
the elements c, c' give a solution of the equation gc = c'g'. □ 
The first k equations of the system Sg^gi form what we call the principal 
system of equations, we denote it by PSg^gi . In what follows we consider 
PSg^gi as a system in variables c, ci,...,Cfc which take values in C, the ele- 
ments pi^p'i, . . . ,Pk,p'k are constants. 

4.2 Regular elements 

Now we specify, in our particular context, the general concepts of a "black hole" 
and "regular part" as discussed in the Introduction. 

Definition 4.4. The set 

BH = {N*g{C) \ C) U Zg{C) 

is called a black hole. Elements from BH are called singular, and elements from 
G \ BH regular. 

Notice that if the subgroup C has a finite malnormality degree in G then 
every element g with lo{g) > mdciG) is regular. In particular, it follows from 
Lemma 12.81 that if G is malnormal in A or in i? then every element g & G with 
l{g) ^ 2 is regular. Notice also, that if g G G \ C is regular then all elements in 
GgC are regular. 

Observe, that the condition 1) in the Conjugacy Criterion, indeed, does not 
apply for regular elements. 

The following description of singular elements follows from Lemma 14.21 
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Corollary 4.5. Let G = A *c B. Then: 

1) an element g E G \ C is singular if and only if the system Bg f. has a 
solution c,ci, . . . ,Ck, where c, Cj are non-trivial elements from C; 

As we have seen already, an element <? € G is singular if and only if the 
system gc = C\g has a nontrivial solution c, c\ in C. 

4.3 Effective recognition of regular elements 

Definition 4.6. Let M be a subset of a group G. \i u,v EG then the set uMv 
is called a G-shift of M. For a set M. of subgroups of G denote by G) 
the least set of subsets of G which contains M. and is closed under G-shifts and 

intersections. 

Lemma 4.7. Let G he a group and C be a subgroup of G. If D E $({G},G), 
Z) ^ then there exist elements gi, . . . ,gn,h G G such that 

D = {C^'n---r)C^")h 

In particular, non-empty sets in $({G}, G) are particular cosets from G. 

Proof. Induction on the number of operations required to construct D from G. 
For a tuple g = {gi, . . . ,gn) of elements from G put 

Cg = (G^i n---nGS"). 

Let D = Cgh for some g e G", h G G. Then for any a,b G G: 

aDb = D"-~\h = Cga-^ab, 

where ga~^ = {gia~^, . . . ,gna~^), i.e., aDb is in the required form. 

Observe, that for arbitrary subgroups K,L ^ G and elements a, 6 G G if 
h€ KaDLb then 

Kar\Lb={Kn L)h. (9) 
Therefore, if /13 e Cg^h\ n Gg-2/12, then 

Cglhl n Gg-2/12 = {Ggl n Cg^)hs = Gg^ks, 

where 53 is concatenation of gi and 52- □ 

Lemma 4.8. Let G = A*c B. Then for given two elements g,g' G G in their 
normal forms 

g = cgpi---pk, g' = cg'p[---p'^. (fc > 1) 

the set Eg^gi , of all elements cinG for which the system PS{g, g') has a solution 
c, ci, . . . , Cfc e C, is equal to 

Eg,g' = c np]:^Gpi n ••• np^^ ■■■p^^Cp[---p'k. 

In particular, if Eg^gi ^ then Eg^gi = Cg^giCg^gi for some subgroup Cg^gi < G 
and some element Cg^gi e G. 
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Proof. Let 

g^cgpi---pk, g' ^ Cg'p[- ■ -p'f,. 

Denote by Vi the set of all solutions (c, ci, . . . , a) £ C*+^ of the system formed 
by the first i equations of PS{g,g'). Let D,n,i be the projection of Vi onto its 
m-s component. 

The first equation of the system PS{g,g') gives: 

PkCaip'k)"^ = ci 

where for uniformity we denote c by cq. Therefore, 

Di,i = PkCip'k)-' n C, Do.i - Pk'D^.ip'k 

and (co, ci) G Vi if and only if 

Cl e Z?!,!, Co ^pl^cip'^. 

Clearly, the sets Do,i and Di.i are in ^c- 

Now we rewrite the «-s equation pfc_i_)-iCi_i — Cip'j,_^j^^ of the system ^^(g, g') 
in the form 

Pk-i+lCi-liPk-i+lY^ = c. 

It follows that 

= Pfc-^+lA-l,^-l(pL.+l)^' n C, (10) 
where i — 1, . . . ,k and I3o.o = C. In particular 

A.fc =Pi£'fc-i.fc-i(pi)"^ nc 

Clearly, (c, ci, . . . , c^) is a solution of the system PS{g, g') if and only if G Dk.k 
and Ci_i = Pfc:':j_,_iCiPfe_j_,_i- More precisely, since 

it follows now that, 

Dk^i^k = Dk^i^k^, np-^Cp'^ n •■• np-^ ■■■p^^Cp[---p'^. 

In particular, 

Egy=Do,k = c npi^Cp'k n ••• npfc^---pr'C'p'i---/fc- 

So Eg^g, e $(C, G). By Lemma SJ] 

PfeiCp'fe n ••• np^^---p^^Cp[---p'k = Hu 

for some subgroup H ^ G and u E G. Now we can see from ([9|) that 

for some subgroup Gg^g' ^ G and Cg.g' G C, as required. □ 
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Denote buy Sub{C) the set of all subgroups of C. By Lemma non-empty 
sets from ^{Sub{C), A) (respectively, from ^{Sub{C), B)) arc some cosets of 
subgroups from A (respectively, from B). 

Corollary 4.9. Let G = A*c B . IJ the Cardinality Search Problem is decidable 
for ^{Sub{C), A) in A and for ^(Sub{C), B) in B then given g, g' as above, one 
can effectively find the set Eg^gi . In particular, one can effectively check whether 
or not Eg^gi is empty, singleton, or infinite. 

Proof. In notations of Lemma 14.81 

^g,g' ^Pk^ ■■ -Pi^Dk^kp'i ■ ■ -p'k- 

Therefore it suffices to solve the cardinality problem for the set -Dfc,fc- The 
quality [Till 

=Pfc-j+iA-i,<-i(pfc-,+i)"^ n C, 

and Lemma H771 show that each is a coset of the type CiCi where Ci ^ C 

and Ci G C. Moreover, since the Cardinality Search Problem is decidable for 
$(S'u6(C), A) in A, and for $(S'u6(C), B) in B, the equahty Q shows how one 
can effectively find the element Ci and the direct expression for the subgroup Ci 
(in terms of shifts and intersections). Therefore, in k steps one can find Dk,k, 
and hence the set Eg^gi. Moreover, on each step one can find the cardinality of 
the set Di^i. This proves the corollary. □ 

Lemma 4.10. Let G = A*cB and g,g' G G. Ifl{g) = l{g') ^ 1 and the system 
PS{g,g') has more then one solution in C then the elements g,g' are singular. 

Proof. Let c, ci , . . . , Cfc and 6, 5i , . . . , 6^ be two distinct solutions of the principal 
system PS{g,g'). Denote for uniformity cq = c,5o = b. Hence we have the 
following systems of equations: 

PkCo = cip'k, pkbo = bip'k 

Pk-lCl = C2Pfc_i, Pk-lh = b2p'k-l 

PiCk-i = Ckp'i, Pibk-1 = bkp'i 

Expressing p'/^ from the first two equations in the system above, and then p'^._i 
from the next two equations, and so on, we get the following equalities: 

ci^^PkCQ = bi^^pkbo 

C2"Vfc-lCl = b2^^Pk-ibi 



Cfc ViCfe-i = bk ^Pibk-i 
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Rewriting these equalities we obtain: 

Pi^bkcl^pi = bk-icl\. 

Observe that all the elements biC~^ are non-trivial. By Lemma [121 the element 
g is singular. Similar argument shows that g' is also singular. □ 

The next result shows that one can effectively determine whether a given 
element g € G \s regular or not. 

Theorem 4.11. Let G ^ A *c B be a free product of finitely presented groups 
A and B amalgamated over a finitely generated subgroup C . Assume that the 
following algorithmic problems are decidable: 

• The Search Membership Problem for the subgroup C in A and in B . 

• The Coset Representative Search Problem for the subgroup G in A and in 
B. 

• The Cardinality Search Problem for ^{Sub{C), A) in A and for ^{Sub{G), B) 
in B. 

• The Membership Problem for N'^{G) and Za{C) in A, and for Ng[C) 
and Zb{C) in B. 

Then there exists an algorithm to determine whether a given element in G is 
regular or not. 

Proof. For a given g G G we can find the normal form of g using Algorithm II. 
Now there arc two cases to consider. 

1) If l{g) > 1 then by Lemma 14.21 a is a singular element if and only if 
the system Bc^g has a nontrivial solution c, ci, . . . , Ck £ G. Observe, that if 
the system Bc^g has two distinct solutions then one of them is non-trivial (i.e., 

C, Ci . . . ,Cfc 1). 

Now if i?c,g has no solutions in C (and we can check it effectively) then g is 
regular. If Beg has precisely one solution then we can find it and check whether 
it is trivial or not, hence we can find out whether g is regular or not. If Bc^g has 
more then one solution (and we can verify this effectively) then g is not regular. 

2) If l{g) — 1 then g € AU B \ G. In this case g is regular if and only if 
g ^ N^{G) U N^{G). Since the sets iV^(C) and N^{G) are recursive one can 
algorithmically check if g is regular or not. 

3) If l{g) — then g is regular if and only if g ^ Za{G). By CoroUarv 14.51 
Zc{G) — Za{G) U Zb{G). Since the sets Za{G) and Zb{G) are recursive one 
can check whether or not g is regular. This proves the theorem. □ 
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Corollary 4.12. Let G = A *c B be a free product with amalgamation of free 
groups A, B. Then the set of regular elements in G is recursive. 

Remark 4.13. The decision algorithm for checking whether a given element 
is regular or not is fast "modulo" Algorithm II and the algorithm B for finding 
cardinality of sets of the type Eg^g^ . In general, both Algorithm II and B can 
be exponential in the worst case. However, we will show later that generically 
both the algorithms are fast. 

One can improve on Theorem 14.111 in the following way. Denote by CR the 
set of all elements in G which have at least one regular cyclically reduced normal 
form of length greater than 1, i.e., CR is the set of elements in G which are 
conjugates of cyclically reduced regular elements. Now by Ci?>i we denote a 
subset of GR consisting of elements of cyclically reduced length grater than 1, 
so Ci?>i is the set of elements in G which are conjugates of cyclically reduced 
regular elements of length greater than 1 . 

Corollary 4.14. Let G — A *c B. Assume that the following algorithmic 
problems are decidable: 

• The Search Membership Problem for the subgroup G in A and B. 

• The Coset Representative Search Problem for the subgroup G in A and B. 

• The Cardinality Search Problem for ^{Sub{C), A) in A and for ^(Sub{C) , B) 
in B. 

• The Membership Problem for (C) and Za (C) in A and for (C) and 
Zb{C) m B. 

Then there exists an algorithm that for a given element g € G decides whether g 
belongs to Ci?>i or not, and if so, then finds a regular cyclically reduced normal 
form of g. 

Proof. Let g £ G. By Lemma 13.81 one can effectively find an element g' G G 
such that g' is a conjugate of g and if l{g') > 1 then g' is a cyclically reduced 
normal form of g. It follows that if l{g') < 1 then g ^ CR. Suppose l{g') > 1. 
We claim that in this case g has a cyclically reduced regular normal form, say 
gi, if and only if at least one of the cyclic permutations of g' is regular. Indeed, 
observe that g' and gi are conjugated in G, hence by the conjugacy criterion 
gi ~ T^i{g'Y fo'^ some ?-cyclic permutation T^i{g') of g' and some c G C. Since 
gi is regular this implies that TTi{g') = gi is also regular (easy calculation). It 
follows that one of cyclic permutations of g' is regular. Now one can effectively 
list all cyclic permutations TTj [g') of g' and apply the decision algorithm from 
Theorem 14.111 to each cyclic permutation TTj{g') to verify if there is a regular 
one among them. This proves the result. 

□ 

Denote by CRo a subset of CR consisting of elements of cyclically reduced 
length 0. 
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Lemma 4.15. Let G = A *c B be a free product of finitely presented groups 
A and B amalgamated over a finitely generated subgroup C . Assume that the 
following algorithmic problems are decidable: 

• The Search Membership Problem for C in A and B . 

• The Coset Representative Search Problem for C in A and B . 

• The Conjugacy Membership Search Problem for C in A and B. 

• The Membership Problem for Za{C) and Zb{C). 

Then there exists an algorithm that for a given element g G G decides whether g 
belongs to GRq or not, and if so, then finds a regular cyclically reduced normal 
form of g. 

Proof. Let g G G. By Theorem [331 one can use Algorithm III to find a cyclically 
reduced normal form oi g. So we may assume from the beginning that g is 
already in a cyclically reduced normal form. If l{g) > then g ^ GRq. Suppose 
l{g) = 0, i.e., g G C. We claim that g G CRq if and only if it is regular. Indeed, 
by conditions of the lemma g is a conjugate of some regular cyclically reduced 
element g' which must be in G (since it has length as g). But then since g' 
is regular the elements g and g' are conjugates in C. Hence g is regular since 
g' is regular. Observe, that g e C is regular if and only if g ^ Zg{C). By 
Corollary [45] Zg(C) = Za{C) U Zb{G). Now, since the Membership problem 
for for Za{C) and Zb{C) is decidable in A and B, one can check algorithmically 
if g belongs to Zq{C) or not, thus solving the question if g is in CRq or not. 

□ 

Remark 4.16. CoroUarv 14.141 claims that the Search Membership Problem for 
the set GRyi is decidable in G = A *c B under corresponding conditions on 
the factors A, B. 

5 The Conjugacy Search Problem and regular 
elements 

In this section we study the Conjugacy Search Problem in a group G ~ A*c B. 

We start with the following particular case of the the Conjugacy Search 
Problem. 

Theorem 5.1. Let G = A *c B be a free product of finitely presented groups 
A and B amalgamated over a finitely generated subgroup C . Assume that the 
following algorithmic problems are decidable: 

• The Word Problem in A and in B. 

• The Search Membership Problem for the subgroup G in A and in B . 
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• The Coset Representative Search Problem for the subgroup C in A and in 
B. 

• The Cardinality Search Problem for ^{Sub{C), A) in A and for ^{Sub{C), B) 
in B. 

Then the Conjugacy Search Problem in G is decidable for all pairs from 

CRyl X G. 

Proof. Let g G Ci?>i and h £ G. By Corollary 14.141 one can find a regular 
cyclically reduced normal form g' oi g. Meanwhile, by Lemma 13.81 one can find 
and element h' £ G such that h' is a conjugate of h and if l{h') > 1 then h' is 
a cyclically reduced normal form of h. Since g £ Ci?>i its cyclically reduced 
length is greater then 1, hence if l{h') < 1 then h is not a conjugate of g. 
Suppose now that l{h') > 1, in this case h' is a cyclically reduced normal form 
of h. This shows that we may assume from the beginning that g is regular and 
g, h are given in cyclically reduced normal forms: 

g = cpi...pk, h = c'p[...pk''- 

According to the conjugacy criterion, the elements g and h are conjugate in G 
if and only if fc = fc' and for some cyclic permutation 7r(/i) of h the equation 
c~^gc — 7r{h) has a solution c in C. By Lemma 14.31 the equation c~^gc = 7r(/i) 
has a solution in C if and only if the system Sg^j^ih) has a solution in G. Since 
g is regular the system PSg .,^(^h-) has at most one solution in C . Decidability 
of the Cardinality Search Problem problems for $(S'u6(C),A) in A and for 
^{Sub{C), B) in B allows one to check whether PSg .,^(j^) has a solution in G or 
not, and if it does, one can find the solution. Now one can verify whether this 
solution satisfies the last equation of the system Sg^T^(ji) or not (using decidability 
of the word problem in A and B). If not, the system Sg^j^i^h) has no solutions 
in C, as well as the equation c^^gc = 7r(/i). Otherwise, the system S'g,7r(h) and 
the equation c~^gc = 7r(/i) have solutions in G and we have found one of these 
solutions. This proves the theorem. □ 

Now we study conjugacy search problem for regular elements of length ^ 1. 

Lemma 5.2. Let G = A *c B be a free product of finitely presented groups 
A and B amalgamated over a finitely generated subgroup G . Assume that the 
following algorithmic problems are decidable: 

• The Search Membership Problem for the subgroup G in A and in B . 

• The Coset Representative Search Problem for the subgroup G in A and in 
B. 

• The Conjugacy Membership Search Problem for G in A and B . 

• The Conjugacy Search Problem in G . 

Then the Conjugacy Search Problem in G is decidable for all pairs from GRqxG. 
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Proof. Let {g, h) G G x G. Since the Membership Search Problem for the set 
CRq is decidable (Theorem 14. ISp one can algorithmically check ii g € GRq, and 
if so, find a regular element g' given in a cyclically reduced normal form which is 
a conjugate of 17. By Theorem l3.9l one can apply Algorithm III to find a cyclically 
reduced normal form of h. Clearly, /i is a conjugate of g if and only if h' is a 
conjugate of g' . Replacing (5, h) by {g' , h') if necessary we may assume that g 
and h are already in their cyclically reduced normal forms and g is regular. It 
follows that g G G since g G GRq. By the conjugacy criterion, ii h ^ G then 
h is not a conjugate of g. li h G C then by the conjugacy criterion there is a 
sequence of elements g = cq, ci, . . . ,Ct — h, where Ci G G and adjacent elements 
Ci and Ci+i, i = 0, . . . , i — 1, are conjugate in A or in B. Since g — cq is regular - 
it does not belong to ZaiG). By Corollary |43] Zg(C) = Za(C) U Zb(C). This 
implies that the element that conjugates cq into ci must be in C. Hence, ci is 
also regular. By induction on t, all the pairs Ci, c^+i are conjugated in C, as well 
as elements g and h. Thus the Search Conjugacy Problem for g and ft. in G is 
reduced to the Search Conjugacy Problem for g and h in G - which is decidable. 

□ 

Lemma 5.3. Let G = A *c B he a free product of finitely presented groups 
A and B amalgamated over a finitely generated subgroup G . Assume that the 
following algorithmic problems are decidable: 

• The Search Membership Problem for the subgroup G in A and in B . 

• The Coset Representative Search Problem for the subgroup G in A and in 
B. 

• The Conjugacy Membership Search Problem for G in A and in B. 

• The Conjugacy Search Problem in A and in B. 

Then the Conjugacy Search Problem in G is decidable for all pairs {g, h) G GxG, 
where g has cyclically reduced length 0. 

Proof. Using Algorithm III (Thcorcm l3.9p one can find cyclically reduced forms 
of a given pair of elements [g, h) G G x G. In particular, one can verify if the 
cyclically reduced length of g and h is equal to 0. If so, then g,h G AU B\G. 
By the conjugacy criterion g, h belong to one and the same factor A or B, and 
they are conjugates there. Since the Search Conjugacy Problem is decidable in 
A and B the last condition is decidable. This proves the lemma. □ 

Remark 5.4. The decision algorithms from Theorem 15. II and Lemmas 15.21 and 
15. 3[ have polynomial time complexity "modulo" the algorithms for finding nor- 
mal forms of elements and the decision algorithms for the problems listed in the 
statements. 

Combining Theorem 15.11 Lemmas 15.21 and 15.31 14.151 and Corollary HUl] alto- 
gether one can get the following general result. 
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Theorem 5.5. Let G = A*c B be a free product of finitely presented groups A 
and B amalgamated over a finitely generated subgroup C. Assume the following 
algorithmic problems are decidable: 

• The Membership Search Problem for C in A and in B . 

• The Coset Representative Search Problem for the subgroup Cin A and B. 

• The Cardinality Search Problem for ^{Sub{C), A) in A and for ^{Sub{C), B) 
in B. 

• The Conjugacy Search Problem in A and in B. 

• The Conjugacy Membership Search Problem for C in A and B . 

• The Membership Problem for N\{C) and Za{C) in A, and Ng{C) and 
Zb{C) in B. 

Then the Conjugacy Search Problem in G is decidable for arbitrary pairs from 
CR X G. 

Corollary 5.6. Let G — A *c B be a free product of free groups A and B 
with amalgamated finitely generated subgroup C . Then the Conjugacy Search 
Problem in G is decidable for arbitrary pairs from GR x G. 

Corollary 5.7. Let G = A*cB and G is malnormal in A. Assume the following 
algorithmic problems are decidable: 

• The Membership Search Problem for G in A and in B . 

• The Coset Representative Search Problem for the subgroup G. 

• The Cardinality Search Problem for ^{Sub{G), A) in A and for ^{Sub{G), B) 
in B. 

• The Conjugacy Membership Search Problem for G in A and B . 

• The Conjugacy Search Problem decidable in A and in B . 
Then the Conjugacy Search Problem is decidable in G. 

Proof. Let (g, h) E G x G. Using Algorithm III (Theorem 13. 9p one can find 
cychcally reduced forms of the elements g, h. Assume for simplicity that g and 
h are cyclically reduced. If their cyclically reduced lengths are not equal, then 
they are not conjugates in G. Therefore we may assume that lo{g) = lo{h). 

1) Suppose l{g) = l{h) > 2. Since G is malnormal in A every element g E G 
with l{g) ^ 2 is regular (see Lemma [2.81) . Hence, in this case by Theorem 15.11 
the Conjugacy Search Problem for every pair {g, h) with l{g) ^ 2 is decidable. 

2) Suppose lo{g) = loih) = 1. In this case the argument from the proof of 
Lemma [531 applies and gives the result. 
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3) Suppose loig) = l{){h) = 0, i.e., g,h € C. By the conjugacy criterion, 
there exists a sequence of elements g = ci,C2, ■ ■ ■ ,Ct = h from C such that the 
neighboring elements are conjugate either in A or in B. By malnormality of 
C in A this implies that the neighboring elements arc. in fact, conjugate in B. 
The latter is algorithmically decidable since the Conjugacy Search Problem is 
decidable in B. 

This proves the corollary. □ 
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